Combining Human and Computer Generated Forecasts Using a Knowledge Based System

1. PREFACE 'Consider mechanically integrating judgmental and statistical forecasts instead of making judgmental adjustments to statistical forecasts …Judgmental adjustment (by humans) of (automatically generated statistical forecasts) is actually the least effective way to combine statistical and judgmental forecasts … (because) judgmental adjustment can introduce bias 1 (Mathews and Diamantopoulos, 1986) …The most effective way to use (human) judgment is as an input to the statistical process … Cleman (1989) reviewed over 200 empirical studies on combining and found that mechanical combining helps eliminate biases and enables full disclosure of the forecasting process. The resulting record keeping, feedback, and enhanced learning can improve forecast quality' (Sanders and Ritzman, 2001). 2. INTRODUCTION Sanders and Ritzman (2001) highlight the difficulty associated with utilising (human) judgment as an input to the statistical process 'when the (human) forecaster gets information at the last minute'. In generating the predictions presented here, the strategy is therefore: • To take judgmental (human) forecasts (derived with the benefit of knowledge of all available computer generated forecast guidance); and, 1 Stern (1996) documents forecaster over-compensation for previous temperature errors. • To input these forecasts into a system that incorporates a statistical process to mechanically combine the judgmental (human) forecasts and the computer generated forecast guidance; Thereby immediately yielding a new set of forecasts. In this context, the purpose of the present work is: 1. To evaluate the new set of forecasts; and, 2. To document the increase in accuracy achieved by that new set of forecasts over that of the judgmental (human) forecasts. Some 30 years ago, Snellman (1977) lamented that whereas the initial impact of guidance material was to increase the accuracy of predictions on account of a healthy human/machine 'mix', operational meteorologists were losing interest and that the gains would eventually be eroded by what he termed the 'meteorological cancer'. Snellman suggested that producing automated guidance and feeding it to the forecaster who 'modifies it or passes it on', encourages forecasters 'to follow guidance blindly' and concluded by predicting an erosion of recent gains. Hindsight informs us from forecast verification statistics that the erosion of gains did not take place. In fact, the accuracy of forecasts continued to increase-see, for example, Stern (2005a, 2005c). Nevertheless, evidence is emerging that the increasing skill displayed by the guidance material is rendering it increasingly difficult for human forecasters to improve upon that guidance Over recent years, the present author has been …